1. Field of the Invention
The present invention relates to data storage and more particularly to systems and methods for achieving data consistency among multiple copies.
2. Description of the Related Art
Many computer applications create multiple copies of the same data. Maintaining consistency of these multiple copies is critically important. How the updating of the different copies is coordinated leads to different levels of consistency among the copies, in return for different costs to perform that coordination. Typically, a stronger consistency, with closer coordination between peer cache updates, results in a larger consumption of resources and larger worst-case completion time.
A problem of keeping multiple caches consistent with each other is evident in processor caches for multiprocessors and file caches for distributed file systems. For processor caches, response times must be extremely fast (orders of magnitude faster than those for Web caches). To achieve these high speeds, the caches have extremely short and fast links of guaranteed reliability to a memory controller that permits them to be informed simultaneously of updates. Techniques that work well given those facilities are simply not practical for distributed applications such as Web caches.
The Andrew File System (AFS) uses a weak consistency method, where the server informs clients of updates. This weak consistency scheme, with the clients checking with the server (see e.g., J. Howard, M. Kazar, S. Menees, D. Nichols, M. Satyanarayanan, R. Sidebotham, and M. West in “Scale and performance in a distributed file system”, ACM Transactions on Computer Systems, 6(1):51-81, February 1988), can have significant overhead.
One problem with implementing applications in prior art consistency schemes emerges from the fact that these schemes are not sensitive to the variations in the consistency needs of an object during its lifetime. These schemes “force” the object to maintain a consistency policy throughout its lifetime. The consistency mechanisms are not flexible enough to adapt to the consistency requirements of the object based on its state.
The effect of this shortcoming is that each object is implemented always with the strongest consistency it would ever require at any point in its lifetime. Maintaining objects at strong consistency levels is costly in terms of the load on both the consistency coordinator and the caches. This heavy load results in the end user experiencing high latencies.
Further, if the load of the consistency management is bursty (as it often is), it will suddenly increase the load on the system. This would not only increase the average latency to unacceptably high values, but might also result in problems such as buffer overflows causing requests to be rejected by the system. Therefore this approach of maintaining strong consistency for all objects does not scale well with the increasing number of objects.
Therefore, a need exists for a scalable and adjustable system and method for providing different levels of consistency throughout a distributed system and for particular objects over their lifetime.